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FOREWORD 


The  Battlefield  Information  Systems  Technical  Area  of  the  U.S.  Army  Research  Institute  for 
the  Behavioral  and  Social  Sciences  (ARI)  does  research  on  tactical  information  systems  - the 
transformation  and  organization  of  battlefield  information,  and  staff  aids  to  battle  management  in 
target  acquisition,  intelligence,  and  command/control  systems  - as  well  as  information  system 
resource  management,  using  automated  tactical  data  systems,  and  the  investigation  of  the  kinds  o 
perceptions  and  thinking  necessary  for  the  proper  judging  of  tactical  intelligence  infornnation.  ARI 
Technical  Paper  278  assessed  the  usefulness  of  traditional  tactical  intelligence  indications  as 
identified  and  interpreted  by  students  in  the  Intelligence  Officers  Advanced  Course;  the  present 
report  describes  a replication  of  the  assessment  with  experienced  intelligence  specialists  from  the 
163d  Military  Intelligence  Battalion. 

The  research  was  done  in  response  to  requirements  of  Army  Project  200621 01 A7W  and  to 
special  requirements  of  the  Combined  Arms  Combat  Development  Activity  (CACDA),  Fort 
Leavenworth,  KS.  _ — ^ 


,J.  E.  UHLANER 
Technical  Director 


THE  PERCEPTION  OF  TACTICAL  INTELLIGENCE  INDICATIONS: 

A REPLICATION 

BRIEF  , 


Requirement: 

To  reassess  findings  from  previous  research  that  brought  into  question  the  effectiveness  of 
"indications"  as  used  in  the  tactical  intelligence  cycle. 


Procedure: 

In  the  present  experiment,  participants  were  28  intelligence  specialists  (officers  and  enlisted 
men),  27  from  the  163d  Military  Intelligence  Battalion  and  1 from  Project  MASSTER. 

Each  specialist  assumed  the  role  of  an  intelligence  staff  analyst  in  the  G-2  section  of  an  Infantry 
Division  conducting  a mobile  defense  in  north-central  West  Germany.  Each  individual  estimated 
the  probability  that  each  of  the  49  indications  of  conventional  military  operations  listed  in  FM 
30-5,  Combat  Intelligence,  would  occur,  given  the  aggressor's  known  course  of  action. 

Each  indication  was  evaluated  with  four  separate  courses  of  action  - Attack,  Defend,  Delay,  and 
Withdraw.  Eleven  indications  were  evaluated  twice  with  each  course  of  action  to  provide  an 
estimate  of  reliability.  The  previous  experiment  had  used  the  same  procedure  with  44  captains 
enrolled  in  the  Intelligence  Officers  Advanced  Course  at  the  U.S.  Army  Intelligence  Center  and 
School,  Fort  Huachuca,  AZ. 


Findings: 

In  both  experiments,  the  estimates  made  by  individuals  were  highly  reliable.  The  variability  in 
the  estimates  made  by  different  individuals  for  the  same  indication  was  extremely  high,  with  an 
average  range  of  estimates  greater  than  .7  on  a 0 to  1 .0  scale. 

Only  12  indications  out  of  49  were  perceived  by  this  group,  and  only  19  by  the  previous 
group,  as  effective  discriminators  of  the  enemy  course  of  action  with  which  they  are  doctrinaliy 
associated. 

There  were  no  significant  differences  between  estimates  made  by  officers  and  by  enlisted 
personnel. 

There  were  no  significant  differences  in  the  estimates  made  by  the  intelligence  specialists 
(officers  and  enlisted  men)  of  the  163d  and  the  estimates  made  previously  by  officers  in  the 
Intelligence  Officers  Advanced  Course. 


ARI  TP  262 
Utilization  of  Findings: 

These  findings  confirmed  the  conclusion  of  the  previous  research  that  traditional  indications  of 
conventional  military  operations  are  either  poorly  understood  or  intrinsically  inadequate  for  use  in 
contemporary  intelligence  operations,  or  both.  Accordingly,  intelligence  personnel  should  be 
extremely  cautious  in  the  use  or  interpretation  of  current  indications. 

Development  of  a user-oriented  indications  structure  could  i^fiprove  the  identification  and 
utilization  of  indications.  Two  possible  approaches  to  such  a structure  are  based  on  the  Delphi 
procedure  of  pooled  expertise  and  on  operational  gaming. 
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THE  PERCEPTION  OF  TACTICAL  INTELLIGENCE  INDICATIONS:  A REPLICATION 


INTRODUCTION 

It  is  virtually  inpossible  for  a military  force  to  conduct  operations 
without  some  telltale  activities.  Identification  and  interpretation  of 
specific  activities  of  a military  force  which  are  typically  associated,, 
with  specific  courses  of  action  - indications  - are  critical  tasks  in 
the  tactical  intelligence  cycle. 

Intelligence  information  is  organized  around  indications  in  the  pro- 
duction of  intelligence;  analysis  of  indications  is  the  basis  for  esti- 
mating aggressor  capabilities  and  vulnerabilities. 

Intelligence  requirements  are  analyzed  in  terms  of  indications 
which,  by  their  existence  or  nonexistence,  provide  an  answer  to  the 
requirement.  Indications  also  provide  the  basis  for  specific  orders  and 
requests  for  information.  Thus,  indications  are  a focal  point  for  both 
the  intelligence  collection  manager  and  the  intelligence  analyst  in 
relating  specific  information  to  intelligence  questions  (Figure  1). 

Recent  research  (Johnson,  Spooner,  and  Jaarsma,  1977)*  suggests  that 
current  indications  of  conventional  military  operations  may  be  inadequate 
for  use  in  contemporary  intelligence  operations. 

An  indication  is  formally  defined  as: 

".  . . any  positive  or  negative  evidence  of  enemy  activ- 
ity or  any  characteristic  of  the  area  of  operations  which 
points  toward  enemy  vulnerabilities  or  the  adoption  or 
rejection  by  the  enemy  of  a particular  capability,  or 
which  may  influence  the  commander's  selection  of  a 
particular  course  of  action."** 

Conditions  which  result  from  either  previous  actions  or  enemy  failure 
to  take  action  may  be  indications.  Indications  are  neither  abstract 
events  nor  specific  data  items.  Rather,  indications  are  patterns  of 
specific  events  or  activities  that  are  consequences  of  conducting  mili- 
tary operations. 

One  such  pattern  is  the  forward  deployment  of  supply  and  evacuation 
installations  in  preparing  to  attack. 


Johnson,  E.  M. , Spooner,  R.  L.  and  Jaarsma.  D.  The  Perception  of 
Tactical  Intelligence  Indications  by  Intelligence  Officers.  ARI 
Technical  Paper  278,  1977. 


★ rk 


Department  of  the  Army  Field  Manual,  FM  30-5,  Combat  Intelligence 
October  1973. 


i 
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Indications  are  probabilistic  in  nature;  they  represent  patterns  of 
activity  which  may  occur  whenever  the  enemy  is  engaged  in  a specific 
course  of  action. 

The  extent  to  which  an  indication  is  an  effective  discriminator  among 
alternative  courses  of  action  depends  on  differences  in  the  relative  fre- 
quency of  occurrence  of  an  indication  with  each  of  the  possible  courses 
of  action. 

if 

For  example,  an  enemy  may  have  deployed  his  combat  elements  in  eche- 
lon on  10%  of  the  occasions  on  which  he  did  not  attack  and  on  80%  of  the 
occasions  when  he  did  attack.  This  activity  then  provides  an  indication 
that  is  highly  predictive  on  an  attack. 

However,  if  an  enemy  had  deployed  his  combat  elements  in  echelon  in 
70%  of  the  occasions  when  he  did  not  attack,  this  activity  would  not  be 
very  predictive  of  the  enemy's  course  of  action. 

Current  tactical  intelligence  indications  were  developed  by  the  U.S. 
Army  in  the  post-World  War  II  period.  These  indications  are  organized 
around  four  general  courses  of  action  - Attack,  Defend,  Delay,  and  With- 
draw (Table  1)  and  emphasize  conventional  infantry,  armor,  and  artillery 
activity  (FM  30-5,  op  cit;  FM  30-102,  1969)* 

Activities  associated  with  the  use  of  nuclear  weapons  and  tactical 
air  have  since  been  treated  as  special  cases.  These  traditional  indica- 
tions are  based  on  lessons  learned  in  World  War  II  and  represent  a sam- 
pling of  typical  indications. 

The  set  is  by  no  means  complete,  nor  is  it  intended  for  dogmatic 
application  in  all  situations,  but  rather  as  a guideline  to  provide  a 
base  from  which  intelligence  personnel  can  develop  indications  appropri- 
ate to  a specific  situation. 

Traditional  indications  are  extensively  used  for  instructional  pur- 
poses in  tactical  intelligence  courses  and  in  field  exercises  and  war 
games.  The  traditional  indications  have  been  criticized  by  intelligence 
personnel  as  being  too  generalized  in  nature  for  most  tactical  situations 
and  too  specialized  as  to  type  of  conflict. 

Many  intelligence  officers  consider  that  the  indications  are  dated 
and  bear  little  relationship  to  contemporary  conflict.  Any  evaluation 
of  the  traditional  indications  must  be  based  in  part  on  military  judgment. 
In  fact,  an  initial  evaluation  could  be  based  solely  on  the  judgment  of 
intelligence  personnel. 


‘k 

Department  of  the  Army  Field  Manual,  FM  30-102,  Handbook  on  Agr;ressor 
Military  Forces.  October  1969. 
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Deployment  of  mechanized  units  on 

good  defensive  terrain  31  (0-100)  78  (30-100)  75  (20-100)  49  (0-100) 
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An  experiment  designed  to  yield  such  an  initial  evaluation  of  the 
effectiveness  of  traditional  indications  vras  recently  conducted  in  terms 
of  the  probability  of  occurrence  of  specific  indications  as  perceived  by 
intelligence  officers  (Johnson,  Spooner,  and  Jaarsma,  op.  cit) . 

In  that  experiment,  44  captains  in  the  Intelligence  Officers  Advanced 
Course  each  assumed  the  role  of  staff  officer  in  the  G-2  section  of  the 
20th  U.S.  Infantry  Division  conducting  a mobile  defense  in  north-central 
West  Germany.  ■* 

Each  officer  estimated  the  probability  of  occurrence  of  each  of  the 
49  indications  of  conventional  military  operations  listed  in  FM  30-5, 
given  a known  aggressor  course  of  action.  Each  indication  was  evaluated 
with  four  separate  courses  of  action  - Attack,  Defend,  Delay,  and  Withdraw. 

Eleven  indications  were  evaluated  twice  with  each  course  of  action  to 
provide  an  estimate  of  reliability.  The  estimates  made  by  each  officer 
were  highly  reliable.  However,  the  variability  in  the  estimates  made  by 
different  officers  for  the  same  indication  was  extremely  high,  with  an 
average  range  of  estimates  greater  than  .7  on  a 0 to  1.0  scale. 

At  most,  19  of  the  49  indications  were  perceived  as  effective  discrim- 
inators of  the  course  of  action  with  which  they  are  doctrinally  associ- 
ated. Further,  the  logic  underlying  clusters  of  related  indications 
could  not  be  clearly  identified  for  any  of  the  four  courses  of  action. 

The  experiment  summarized  above  raised  serious  doubts  as  to  the  adequa- 
cy of  current  tactical  intelligence  indications  and  suggested  that  criti- 
cisms by  intelligence  personnel  are  valid.  Several  possible  explanations 
were  suggested  to  account  for  the  results: 

1.  The  variability  in  the  perceived  probability  of  occurrence  of  the 
indications  accurately  reflects  their  relationship  witn  the  alternative 
courses  of  action,  and  the  indications  are  intrinsically  inadequate. 

2.  The  indications  are  effective  discriminators  between  alternative 
courses  of  action,  but  the  officers  did  not  possess  the  requisite  knowl- 
edge to  evaluate  the  indications. 

3.  It  is  not  feasible  to  evaluate  indications  in  the  context  of  a 
brief  description  of  the  situation,  and  doctrine  concerning  the  general- 
ity of  indications  is  misleading. 

4.  It  is  not  feasible  to  evaluate  indications  singly,  and  the  only 
doctrinal  basis  for  grouping  indications  - in  terms  of  the  four  courses 
of  action  - is  Inadequate. 


V'/hich  of  these  explanations,  or  combination  of  explanations, 
accounted  for  the  results  of  Johnson,  Spooner,  and  Jaarsma  could  not 
be  determined  from  the  data. 
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A fifth  possible  explanation  of  the  data,  however,  is  that  all  the  | 

intelligence  officers  serving  as  subjects  were  not  oriented  toward  tacti- 
cal intelligence.  Many  military  intelligence  officers,  although  familiar  j 

with  tactical  Intelligence,  are  primarily  trained  and  experienced  in 
other,  different  areas  of  intelligence,  e.g.,  counterintelligence. 

Intelligence  personnel  trained  in,  and  assigned  to,  tactical  intelli- 
gence positions  may  perceive  the  current  tactical  intelligence  indica- 
tions differently  than  the  "average"  intelligence  officer.  This  alter- 
native explanation  could  be  evaluated  by  replicating  the  previous  experi- 
ment using  only  tactical  Intelligence  personnel. 

Such  a sample  of  tactical  intelligence  personnel  was  made  available  , 

through  the  cooperation  of  Project  MASSTER  (Modern  Army  Selected  Systems 
Test  Evaluation  and  Review)*  and  the  163d  Military  Intelligence  Battalion 
stationed  at  Fort  Hood,  TX. 

The  specific  objective  of  the  present  replication  was  to  compare  the 
perception  of  tactical  intelligence  Indications  by  personnel  from  the 
163d  Military  Intelligence  Battalion  with  data  previously  obtained  from 
students  in  the  Intelligence  Officers  Advanced  Course. 


METHOD 

Sample 

Participating  in  the  experiment  were  28  intelligence  specialists 
stationed  at  Fort  Hood,  TX.  One  was  from  Project  MASSTER,  Experimental 

Task,  and  the  other  27  were  members  of  the  163d  Military  Intelligence  i 

Battalion. 

Each  participant  was  asked  to  assume  the  role  of  an  intelligence 
staff  analyst  in  the  G-2  section  of  the  20th  U.S.  Infantry  Division 
conducting  a mobile  defense  in  north-central  West  Germany  near  the 
east/west  German  border. 

In  the  context  of  the  scenario  described  below,  participants  esti- 
mated the  probability  that  each  of  49  indications  would  occur,  given  a 
known  aggressor  course  of  action.  Each  indication  was  evaluated  relative 
to  four  enemy  courses  of  action  - Attack,  Defend,  Delay,  Withdraw. 


i 


is 

Now  TCATA  (TRADOC  Combined  Army  Test  Activity). 
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Scenario.  A brief  scenario  was  presented  in  which  each  of  four 
courses  of  action  was  feasible  for  the  Aggressor.  Following  a 
coordinated  non-nuclear  attack  into  West  Germany  in  July,  NATO  forces 
had  blunted  the  attack,  and  the  front  had  become  relatively  stable  by 
15  August. 

The  Aggressor's  main  penetration  was  north  of  the  Sauer land.  South 
and  east  of  the  Sauerland,  NATO  forces  were  in  defensive  positions  east 
of  the  Vogelsberg.  As  of  15  September,  the  G-2  Intelligence  Estimate  v 
listed  the  following  Aggressor  capabilities: 

1.  Continue  defense  along  the  border  area. 

2.  Attack  at  any  time  along  the  border. 

3.  Withdraw  major  elements  of  the  2nd  CAA  to  support  operations  in 
the  North. 

4.  Conduct  delaying  operations  along  the  border  while  main  elements 
move  to  defensive  positions. 

Indications . From  FM  30-102,  49  indications  descriptive  of  enemy 
activity  were  selected.  These  49  included  all  indications  relating  to 
mid- intensity  non-nuclear  conflict,  as  well  as  indications  of  nuclear 
operations  related  to  non-nuclear  operations. 

By  doctrine,  30  of  the  49  indications  are  related  to  Attack,  15  to 
Defend,  3 to  Delay  and  1 to  Withdraw.  Two  indications  of  Reinforcement 
and  four  of  Nuclear  operations  are  Included  in  the  30  Attack  indications. 


/ 


Procedure 

A response  booklet  was  prepared  which  included  the  scenario,  instruc- 
tions, and  lists  of  indications.  This  booklet  was  identical  with  the 
response  booklet  used  in  the  previous  experiment.  For  each  course  of 
action,  four  different  pages  of  15  indications  each  were  prepared. 

Eleven  indications  were  listed  twice  with  each  course  of  action. 


The  60  Indications  were  listed  in  random  order  for  each  couse  of 
action.  However,  an  Indication  was  never  listed  twice  on  the  sane  page. 
The  four  pages  of  indications  for  each  Aggressor  course  of  action  were 
assembled  in  one  of  24  possible  orderings. 

Thus,  each  subject  evaluated  all  of  the  indications  relative  to  one 
course  of  action  before  proceeding  to  evaluate  the  indications  given 
another  course  of  action.  The  four-page  sets  of  indications  for  each  of 
the  four  Aggressor  courses  of  action  were  assembled  in  one  of  24  possible 
orderings. 
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There  were  576  possible  unique  orderings  of  indications  and  courses 
of  action  in  the  response  booklets.  Each  subject  evaluated  the  16  pages 
of  indications  in  a different  order. 

The  experinent  was  conducted  in  a single  session  in  a large  conference 
room.  After  the  response  booklets  were  handed  out,  an  overview  of  the 
experiment  was  read,  and  subjects  were  allowed  10  minutes  to  become 
familiar  with  the  scenario.  They  were  then  instructed  that 

. . .Your  task  on  the  16  pages  of  the  Indications 
section  is  to  evaluate  for  each  indication  listed 
the  probability  that  it  will  occur,  given  the 
Aggressor  taking  the  specific  course  of  action 
listed  at  the  top  of  the  page.  These  evaluations 
are  to  be  made  in  the  context  of  the  Meiningen  Gap 
Scenario  given  in  the  previous  section.  You  should 
not  concern  yourself  with  the  specific  details  of 
the  scenario.  Rather,  you  should  consider  it  as 
a general  framework  within  which  to  make  your 
evaluations. 

The  way  you  should  think  of  your  assessment 
is  as  a probability  of  occurrence.  Since  proba- 
bilities range  from  0 to  1,  this  is  the  range  of 
assignments  you  may  use  to  describe  your  judgment. 

This  may  be  confusing  so  let's  look  at  a similar 
type  of  judgment  - weather  predicting  ... 

Subjects  were  instructed  to  evaluate  each  indication  independently  of  any 
other  indication,  considering  only  the  Aggressor's  true  course  of  action, 
to  complete  each  page  before  continuing  to  the  following  page,  and  not 
to  refer  back  to  a page  once  it  had  been  completed.  It  was  stressed  that 
accuracy  was  more  important  than  speed.*  Breaks  were  allowed  as  required. 
Subjects  required  40  to  75  minutes  to  complete  the  task  and  to  complete 
an  experience  questionnaire. 


Other  methods  are  available  for  eliciting  more  "precise"  subjective 
probability  estimates  (Huber,  G.  P.  Methods  For  Quantifying  Subjec- 
tive Probabilities  and  Multi-Attribute  Utilities.  Decision  Sciences. 
1974,  _5,  430-458).  These  methods  all  require  that  a consultant  work 
directly  with  the  individual  making  the  estimate.  Thus,  although  not 
appropriate  for  the  present  study,  the  methods  would  be  appropriate 
in  developing  baseline  data. 
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RESULTS 

The  basic  data  were  the  probabilities  of  occurrence  assigned  by 
each  subject  to  each  indication,  given  each  of  the  four  courses  of 
action.  Of  principal  interest  was  a comparison  of  the  perception  of 
indications  by  the  intelligence  specialists  of  the  163d  with  data 
previously  collected  from  officers  in  the  Intelligence  Officers  Advanced 
Course. 

Ik 

The  subjects  in  the  ttwo  groups  are  described  below.  The  first  ques- 
tion considered  was  the  reliability  of  the  estimates  provided  by  the 
subjects.  Analyses  of  two  types  are  then  presented: 

One  focuses  on  differences  among  specific  Indications  considering 
only  the  subjects  in  the  present  experiment,  while  the  other  focuses  on 
differences  between  the  two  groups  of  subjects  - intelligence  special- 
ists and  students  in  the  Intelligence  Officers  Advanced  Course. 


Subject  Profile 

Two  of  the  28  subjects  in  the  present  experiments  did  not  complete 
the  post-experiment  questionnaire.  The  profile  therefore  is  based  on 
26  subjects.  There  were  13  officers  (one  0-1,  four  0-2s , six  0-3s,  and 
two  0-5s)  and  13  enlisted  men  (two  E-2s , one  E-4,  six  E-6s , four  E-7s, 
and  one  E-8). 

The  officers  had  1 to  19  years  of  active  duty;  the  average  was  7.5 
years.  The  enlisted  men  had  8 months  to  20  years  of  active  duty;  the 
average  was  9.6  years.  Primary  Military  Occupational  Specialties  (PMOS) 
among  the  officers  were  distributed  as  follows: 

Eight  Tactical  Intelligence  Staff  Officers  (PMOS  9301) 
with  six  months  to  three  years'  experience;  the  average 
was  1.7  years. 

Two  Counterintelligence  Officers  (PMOS  9666),  each  with 
two  years'  experience.  One  of  these  officers  had  12 
previous  years'  experience  as  a Tactical  Intelligence 
Staff  Officer. 

Two  Strategic  Intelligence  Staff  Officers  (PMOS  9300), 
one  with  one  and  the  other  with  six  years'  experience. 

One  of  these  officers  had  four  previous  years'  experi- 
ence as  a Tactical  Intelligence  Staff  Officer. 

One  Aerial  Surveillance  Officer  (PMOS  9309)  with  seven 
years'  experience. 
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All  the  enlisted  personnel  were  Intelligence/Order  of  Battle  Ana- 
lysts (PMOS  96B)  with  three  months  to  20  years'  experience;  the  average 
experience  in  intelligence  was  5.4  years. 

The  subjects  considered  themselves  familiar  with  indications,  and 
most  (20  of  the  26)  said  that  they  had  used  the  standard  intelligence 
indications  in  operational  settings,  field  exercises,  or  both. 

These  subjects  represent  a much  more  skilled  and  experienced  group* 
of  tactical  intelligence  personnel  than  those  in  the  previous  research. 
All  44  students  in  the  original  experiment  were  captains  with  1 to  11 
years  of  active  duty;  the  average  was  6.3  years. 

Primary  Military  Occupational  Specialties  were  varied,  with  only  two 
PMOS  represented  by  5 or  more  students;  17  were  Counterintelligence 
Officers  (PMOS  9666)  and  6 were  Tactical  Intelligence  Staff  Officers 
(PMOS  9301).  The  students  were  in  the  fifth  month  of  the  Intelligence 
Officers  Advanced  Course  and  had  recently  received  the  block  of  instruc- 
tion concerning  collection  planning  and  the  use  of  indications. 

Post-experiment  questionnaires  showed  that  the  students  considered 
themselves  familiar  with  indications.  However,  most  had  never  used 
indications  in  an  operational  setting. 


P>,esponse  Reliability 

The  11  indications  presented  twice  with  each  given  course  of  action 
were  pooled  over  the  four  courses  of  action  to  provide  a total  of  44 
repeated  estimates  by  each  subject.  The  mean  intrasubject  product- 
moment  correlation  for  the  28  subjects  was  .79,  with  a range  of  .42 
to  .97  and  a standard  deviation  of  .12. 

These  coefficients  reveal  that  subjects  were  quite  consistent  in 
their  perceptions  of  the  probability  of  occurrence  of  specific  indica- 
tions. Subsequent  analyses  used  the  mean  of  a subject's  two  responses 
for  each  of  the  11  indications  repeated  within  a given  course  of  action. 

For  subjects  in  the  previous  experiment,  the  mean  intrasubject 
product-moment  correlation  was  .77,  with  a range  of  .39  to  .97  and  a 
standard  deviation  of  .13.  The  small  difference  between  the  groups  in 
mean  correlation  is  not  significant. 
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Differences  among  Indications  - Intelligence  Specialists 

The  mean  and  range  of  the  perceived  probability  of  occurrence  of  each 
indication  for  each  course  of  action  are  presented  in  Table  1.  (The 
corresponding  standard  deviations  averaged  23.4,  varying  from  12.3  to 
34.3*). 

All  indications  had  a non-zero  mean  perceived  probability  of  occur- 
rence with  each  of  the  four  courses  of  action.  That  is,  all  Indications 
were  perceived  as  feasible,  given  any  of  the  four  courses  of  action. 

The  intersubject  variability  of  the  estimates  was  extremely  high,  with 
an  average  range  of  87.9. 

Note  that  the  subjects  did  not  use  the  fine-grain  feature  of  the  100- 
point  scale  in  making  their  judgments.  All  but  one,  of  more  than  5,000 
separate  assessments  by  the  28  individuals,  was  made  in  multiples  of  10, 
i.e.,  20,  30,  etc. 

An  analysis  of  variance  of  indications  x courses  of  action  revealed 
significant  differences  between  courses  of  action  (F(3,81)  = 27.2, 
p < .001),  indications  (F(48,1296)  = 9.69  p < .001),  and  a significant 
interaction  of  indications  x courses  of  action  (F(144,3888)  = 19.58, 

p < .001). 

Comparisons  among  the  four  courses  of  action  averaged  over  the  49 
indications  are  not  very  useful.  Neither  are  comparisons  among  the  49 
Indications  averaged  over  the  four  courses  of  action.  Rather,  the 
comparisons  of  interest  concern  the  interaction  among  courses  of  action 
for  a given  indication. 

Results  of  the  comparisons  using  the  Tukey  (b)  procedure  (Winer, 
1952**)  at  a .01  level  of  significance  are  shown  in  Table  1.  Entries 
underlined  by  a common  line  do  not  differ  significantly  from  each  other; 
entries  not  so  underlined  do  differ  significantly. 

Using  indication  Number  1 as  an  example,  the  mean  perceived  probabil- 
ity of  occurrence  was  significantly  higher  given  Attack  than  given  either 
Defend,  Delay,  or  Withdraw.  The  mean  perceived  probability  of  occurrence 
was  not  significantly  different  when  given  either  Defend,  Delay,  or 
Withdraw. 


:1c 

Previous  data  from  the  44  students  on  the  mean  and  range  of  the 
perceived  probability  of  occurrence  of  each  indication  for  each 
course  of  action  are  presented  in  Table  2.  (The  corresponding 
standard  deviations  averaged  20.4,  varying  between  11.8  and  26.0). 

Winer,  B.  J.  Statistical  Principles  in  Experimental  Design.  Nev?  York: 
McGraw-Hill,  1962. 
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For  a few  indications,  the  pattern  of  significant  differences  may  not 
be  iomediately  clear.  The  complexity  results  from  listing  the  courses  of 
action  in  a standard  order,  rather  than  in  terms  of  an  ascending  or 
descending  order  of  mean  perceived  probability  of  occurrence. 

The  pattern  of  significant  differences  becomes  apparent  if  the  mean 
perceived  probability  of  occurrence  associated  with  each  course  of  action 
is  listed  in  a descending  order  as  shown  below  for  indication  Number  16. 

Attack  Delay  Withdraw  Defend  ^ 

74  55  45  44 


The  mean  perceived  probability  of  occurrence  was  not  significantly  differ- 
ent between  the  Aggressor  courses  of  action  of  Attack  and  Delay,  or  among 
Delay,  Withdraw,  and  Defend.  However,  the  indication  was  perceived  as 
significantly  more  probable  if  the  Aggressor  course  of  action  was  Attack 
than  if  the  Aggressor  course  of  action  was  Withdraw  or  Defend. 

Results  of  the  comparisons  shown  in  Table  1 can  be  summarized  as 
follows ; 

1.  For  14  of  the  49  indications  (11  Attack  and  3 Defend)  there  were 
no  significant  differences  in  the  mean  perceived  probability  of  occur- 
rence among  the  four  courses  of  action. 

1 2.  Eleven  of  the  30  indications  doctrinally  associated  with  Attack 

I had  significantly  higher  mean  perceived  probabilities  of  occurrence  given 

Attack  than  given  any  of  the  remaining  three  courses  of  action.  These 
indications  are  identified  by  an  asterisk  (*)  in  Table  1. 

3.  One  of  the  15  indications  doctrinally  associated  with  Defend  had 
a significantly  higher  mean  perceived  probability  of  occurrence  given 
Defend  than  given  any  of  the  remaining  three  courses  of  action.  This 
indication  is  also  identified  by  an  asterisk  (*)  in  Table  1. 

»■ 

t 4.  Neither  the  three  indications  doctrinally  associated  with  Delay 

I nor  the  indication  of  Withdraw  had  significantly  higher  mean  perceived 

probabilities  of  occurrence  given  Delay  or  Withdraw,  respectively. 

Results  of  the  comparison  tests  imply  that  traditional  indications 
are  not  good  discriminators  among  the  four  courses  of  action.  Only  12 
j i,  of  the  49  indications  (11  Attack  and  1 Defend)  had  a significantly 

higher  mean  perceived  probability  of  occurrence  given  the  course  of 
'■  action  with  which  they  are  doctrinally  associated. 
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Considering  only  these  12  indications,  given  the  doctrinal  course  of 
action,  the  perceived  probability  of  occurrence  had  a range  of  at  least 
.50,  and  a mean  perceived  probability  of  occurrence  of  at  least  .16  with 
some  other  course  of  action.  Thus,  given  an  estimate  from  an  individual 
using  one  of  these  12  "good"  indications,  the  Aggressor's  course  of 
action  is  still  somewhat  ambiguous. 

One  possible  source  of  variation  in  this  sample  is  the  differential 
training,  experience,  and  interests  between  officers  and  enlisted  person- 
nel. This  factor  was  evaluated  using  the  26  subjects  who  completed  the 
post-experiment  questionnaire. 

An  analysis  of  variance  of  type  of  personnel  x indications  x courses 
of  action  revealed  no  significant  differences  between  the  13  officers  and 
the  13  enlisted  personnel  (F(l,24)  = 2.3665),  nor  any  significant  inter- 
actions with  type  of  personnel. 

Comparison  Between  Students  in  the  Intelligence  Officers  Advanced  Course 
and  Intelligence  Specialists  (officers  and  enlisted  men)  of  the  163d 

Individuals  in  both  groups  were  highly  consistent  in  their  own  percep- 
tions of  the  probability  of  occurrence  of  specific  indications.  There 
was  no  significant  difference  in  the  mean  individual  reliability  of 
estimation  between  the  two  groups. 

There  was  a high  degree  of  similarity  in  the  mean  and  range  (and 
standard  deviations)  of  the  perceived  probability  of  occurrence  of  each 
indication  for  each  course  of  action  between  the  two  groups  (Table  1 and 
Table  2. 

An  analysis  of  variance*  of  subject  group  x indications  x course  of 
action  revealed  no  significant  main  effect  of  subject  groups  (F(l,70)  = 
.6).  However,  there  were  two  significant  interactions  with  subject  group: 
subject  group  x indications  (F(48,3260)  = 10.91)  and  subject  group  x 
indications  x course  of  action  (F(144,  10080)  = 24.19).  As  discussed 
previously,  comparisons  between  groups  for  each  indication  averaged  over 
the  four  courses  of  action  would  not  be  very  useful.  Comparison  between 
groups  for  each  indication  with  each  course  of  action  were  made  using 
Tukey  (b)  procedure  at  a .01  level  of  significance. 

Of  the  196  possible  between-group  comparisons  for  each  indication 
with  each  course  of  action,  only  15  were  significant.  There  appeared  to 
be  no  pattern  to  the  differences  between  groups  (Tables  1 and  2). 


An  unweighted-means  solution  was  used  in  view  of  the  unequal  number  of 
subjects  in  the  two  groups  (Winer,  1962,  op.cit). 


© 


Table  2.  Mean  and  Range  of  Perceived  Probability  of  Occurrence  of  Standard  Intelligence 
Indications  Estimated  by  44  Students  in  the  Intelligence  Officers  Advanced  Course 
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(From  Johnson,  Spooner,  and  Jaarsma,  1977) 
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(From  Johnson,  Spooner,  and  Jaarsma,  1977) 
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For  example,  in  8 of  15  significant  comparisons,  students  in  the 
Intelligence  Officers  Advanced  Course  assigned  a significantly  higher 
mean  probability  of  occurrence  than  did  personnel  in  the  present  experi- 
ment. 

For  the  remaining  seven  significant  comparisons,  students  in  the 
Intelligence  Officers  Advanced  Course  assigned  a significantly  lower 
mean  probability  of  occurrence.  Significant  interactions  Involving 
subject  groups  appear  to  result  from  idiosyncratic  rather  than  systematic 
differences  between  groups  in  the  perception  of  indications. 


Correlation  among  Indications 

None  of  the  indications  proved  unique,  nor  would  one  so  expect 
then  - the  reason  indications  rarely  are  used  singly.  This  finding 
suggests  that  patterns  of  intercorrelations  between  indications  nay 
define  clusters  or  groups  of  related  indications  associated  in  a partic- 
ular way  with  a given  course  of  action. 

In  the  previous  experiment  based  on  students  in  the  Intelligence 
Officers  Advanced  Course,  principal  components  factor  analysis  was  used 
to  abstract  the  underlying  or  implicit  dimensions  of  variation  in  the 
assessments  of  the  indications  for  each  course  of  action.  Two  findings 
emerged  from  those  analyses: 

1.  Indications  were  perceived  as  sets  of  related  events  having  some 
overall  structure,  not  as  single  events  or  activities. 

2.  The  logic  underlying  clusters  of  related  Indications  could  not 
be  identified  clearly  for  any  of  the  four  courses  of  action.  In  view  of 
great  similarity  between  data  from  the  present  experiment  and  data  from 
the  previous  research  on  which  the  above  findings  were  derived,  those 
analyses  have  not  been  repeated. 

For  a more  detailed  discussion  of  the  previous  findings,  refer  to 
Johnson,  Spooner,  and  Jaarsma,  op.  cit. 


IMPLICATIONS  OF  THE  RESULTS 

The  most  significant  finding  of  the  present  replication  is  the 
close  similarity  in  the  perceived  frequency  of  occurrence  of  indications 
between  intelligence  specialists  (officers  and  enlisted  men)  of  the  163d 
Military  Intelligence  Battalion,  and  officers  attending  the  Intelligence 
Officers  Advanced  Course. 
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Although  individual  evaluations  of  a given  indication  were  highly 
reliable,  there  was  little  agreement  among  individuals  concerning  the 
perceived  frequency  of  occurrence  of  an  indication  with  each  of  the 
four  courses  of  action. 

Few  indications  were  perceived  as  effective  discriminators  of  the 
course  of  action  with  which  they  are  doctrinally  associated.  Thus,  the 
standard  indications  are  likely  ineffective,  either  as  the  focus  of 
analysis  or  as  the  basis  for  collection  planning.  ~ 

The  present  experiment  represents  an  initial  evaluation  of  indica- 
tions based  solely  on  military  judgment.  The  findings  derive  from  the 
lack  of  agreement  among  individuals  concerning  the  perceived  frequency 
of  occurrence  of  an  indication,  in  association  with  each  of  the  four 
courses  of  action,  and  not  from  any  reference  to  the  validity  of  specific 
indications. 

Apparently,  both  the  doctrine  associated  with  indications,  and  the 
training  in  identification  and  use  of  indications,  should  be  systemat- 
ically examined  and  new  or  remedial  methods  developed.  Further,  it  is 
clear  that  valid  doctrine  must  be  available  before  training  can  be 
improved. 

The  methodology  employed  in  both  experiments  was  based  on  improved 
collection  planning  methods  being  developed  (Jaarsma,  Spooner,  and  Johnson, 
1977*).  Estimates  of  the  probability  of  occurrence  of  an  indication, 
given  a known  aggressor  course  of  action,  are  the  reverse  of  the  logic 
normally  employed  by  an  analyst. 

An  analyst  usually  would  estimate  the  probability  of  occurrence  of 
an  aggressor  course  of  action  given  a known  indication.  This  "reverse 
logic"  itself  did  not  appear  a significant  determinant  of  the  results, 
as  individual  estimates  in  both  this  and  the  previous  experiment  were 
highly  reliable. 

A more  important  feature  of  the  methodology  is  that  it  was  knowledge- 
based  rather  than  performance-based.  That  is,  individuals  were  not 
required  to  perform  an  analysis  or  planning  task,  but  to  estimate  rela- 
tionships. A methodology  that  is  performance-based  might  yield 
different  results. 


Jaarsma,  D. , Spooner,  R.  L. , and  Johnson,  E.  M.  Methods  for  Planning 
Intelligence  Collection:  Acquisition  Priorities  and  P^esource  Allocation. 

ARI  Technical  Paper,  1977. 
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However,  findings  of  this  and  the  previous  experiment  are  so  consis- 
tent that,  rather  than  develop  improved  procedures  to  evaluate  tradi- 
tional indications  for  use  within  collection  planning  or  in  analysis, 
it  would  be  better  to  focus  research  on  Improving  the  identification 
and  utilization  of  indications. 

Such  an  approach  would  not  only  provide  a more  detailed  evaluation 
of  current  indications,  but  would  also  lead  to  improved  indications  and 
associated  doctrine.  ^ 

Two  areas  of  development,  which  could  improve  identification  and 
utilization  of  indications,  were  suggested  by  Johnson,  Spooner,  and 
Jaarsma*.  One  area  is  development  of  an  improved  indications  structure; 
the  other  is  development  of  baseline  data  representing  the  probability 
of  occurrence  of  specific  indications  for  specific  courses  of  action. 

The  potential  advantages  of  development  in  these  two  areas  are 
numerous.  However,  both  analytic  and  empirical  research  is  required  if 
indications  are  to  be  a generally  useful  and  effective  technique  in 
intelligence  operations. 

The  development  of  an  adequate  framework  or  structure  for  Indications 
must,  of  necessity,  precede  the  development  of  baseline  data.  Two 
approaches  which  could  be  used  to  develop  a user-oriented  indications 
structure  are  the  Delphi  technique  and  operational  gaming. 

The  Delphi  procedure  (Linstone  and  Turoff,  1975**)  involves  the 
sequential  individual  questioning  (usually  by  questionnaire)  of  a set 
of  experts,  interspersed  with  information  and  opinion  feedback  derived 
from  earlier  parts  of  the  questionnaire  program. 

The  Delphi  technique  differs  from  the  usual  survey  or  questionnaire 
in  that  participants  are  not  identified  in  succeeding  phases,  but  remain 
anonjTnous.  A Delphi  procedure  for  indications  might  begin  by  attempting 
to  identify  specific  indications  and  their  associated  specific  informa- 
tion requirements,  within  the  context  of  a specific  scenario. 

The  procedure  might  begin  at  a specific  time  in  thi  course  of  the 
scenario  and  examine  successive  phases  of  sn  engagement  as  agreement 
among  the  experts  is  reached  within  each  phase.  A participant  in  the 
Delphi  would  be  asked  to  assume  either  the  role  of  an  intelligence  staff 
analyst  or  that  of  the  enemy  commander. 


*Ibid. 


Linstone,  H.  A.,  and  Turoff,  M.  (Eds.).  The  Delphi  Method:  Techniques 

and  Applications.  Reading,  MA;  Addison-Wesley , 1975. 
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Individuals  in  the  analyst  role  would  attempt  to  specify  what  indica- 
tions might  be  present  and  what  data  would  be  required  to  confirm  or  deny 
each  indication.  Individuals  in  the  enemy  commander  role  would  attempt 
to  specify  the  specific  actions  they  would  take  to  pursue  certain  courses 
of  action  and  what  "traces"  would  be  available  from  each  of  these  actions. 

After  each  round  of  questions,  responses  would  be  consolidated  into 
a unified  response  by  the  experimenters,  highlighting  areas  of  both 
agreement  and  disagreement  among  the  participants.  ^ 

This  information  would  be  fed  back  to  the  participants  as  part  of 
the  next  round  of  questions.  This  cycle  would  continue  until  the 
participants  made  no  change  in  response  as  a result  of  feedback,  and  the 
Delphi  would  then  move  to  the  next  phase  of  the  scenario. 

Operational  gaming  (Shubik,  1975*)  could  be  conducted  in  a similar 
manner  to  develop  an  indications  structure,  the  major  exception  being 
that  the  game  actually  would  be  played  in  accord  with  all  environmental 
and  resource  constraints  of  the  military  forces  in  the  scenario. 


Thus,  individuals  in  the  role  of  an  enemy  commander  would  determine 
which  actions  would  be  taken  in  pursuing  a course  of  action,  and  individ- 
uals in  the  analyst  role  would  attempt  to  identify  specific  actions  and 
their  associated  data  in  order  to  determine  the  course  of  action  being 
pursued. 

Tabletop  or  board  games  would  be  ideal  for  use  in  an  analysis  of 
indications,  since  precise  estimates  of  combat  outcomes,  although  desir- 
able, are  not  required.  This  type  of  game  can  provide  flexible  and 
detailed  play  in  the  command,  control,  and  Intelligence  areas.  The  game 
can  be  stopped  at  any  point  to  allow  analysis  by  the  participants.  The 
procedure  is  relatively  inexpensive. 

The  objectives  of  an  analysis  of  indications  based  on  either  a Delphi 
procedure  or  on  operational  gaming  are  similar,  although  the  results 
obtained  would  probably  differ. 

The  Delphi  has  the  advantage  that  the  knowledge  and  expertise  of  a 
large  number  of  intelligence  and  operations  experts  can  be  pooled, 
minimal  resources  and  time  are  required,  and  there  is  little  interference 
with  the  normal  assigned  duties  of  participants. 
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A Delphi  procedure,  however,  provides  no  direct  method  for  evaluating 
its  findings.  Operational  gaming,  on  the  other  hand,  has  the  advantage 
of  providing  a built-in  evaluation  as  successive  phases  of  the  scenario 
are  played. 

Operational  gaming  required  more  resources  than  Delphi,  would  involve 
fewer  experts,  and  would  require  a large  amount  of  dedicated  time  from 
participants.  The  two  approaches  are  not  mutually  exclusive,  and  one 
approach  would  benefit  from  the  results  of  the  other. 

A recommended  strategy  for  developing  an  improved  indications 
structure  would  be  to  conduct  a Delphi  procedure  followed  by  operational 
gaming.  The  Delphi  could  be  used  to  identify  specific  Indications, 
relationships  among  Indications,  and  associated  data  requirements  for 
a scenario  of  the  type  used  in  operational  gaming. 

The  Delphi  would  provide  an  initial  structure  for  the  operational 
gaming  as  well  as  identify  some  of  the  characteristics  required  in  the 
gaming  methodology  in  order  to  analyze  Indications. 

Flexible  and  detailed  play  in  the  command,  control,  and  intelligence 
areas  is  seldom  Incorporated  in  gaming,  and  having  the  results  of  the 
Delphi  before  initiating  the  operational  gaming  should  serve  both  to 
simplify  the  task  and  to  enhance  the  results  of  gaming. 

This  two-fold  strategy  capitalizes  on  the  unique  advantages  of  both 
approaches  - the  Delphi  for  tapping  a large  number  of  experts,  and 
operational  gaming  for  detailed  analysis  and  evaluation. 

Once  a valid  indications  structure  has  been  developed,  a number  of 
existing  techniques  can  be  used  to  develop  baseline  data  representing 
probability  of  occurrence  of  specific  indications  as  well  as  relation- 
ships between  indications  for  specific  courses  of  action  (Huber,  1974, 
op.  cit) . 

Validity  and  effectiveness  of  such  baseline  information  would  depend 
on  the  quality  of  the  indications  structure. 

Indications  constitute  a potentially  powerful  tool  for  intelligence 
personnel.  However,  extreme  caution  should  be  exercised  in  the  use  and 
interpretation  of  current  indications. 

There  is  no  easy  and  simple  solution  to  the  problems  surfaced  by  the 
results  of  this  and  the  previous  experiment.  Both  analytic  and  empirical 
research  is  required  if  indications  are  to  be  a generally  useful  and 
effective  technique  in  intelligence  operations. 
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